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What is claimed is: 



1 LA system for interacting with displays and all devices that use 

2 such displays comprised of 

3 a. a display, 

4 b. a sensor or camera, 

5 c. a pointing device that can be registered by the sensor or 

6 camera, 

7 d. a method for detecting the pointing device, 

8 e. a method for establishing the mapping between the position of 

9 the pointing device and a corresponding location on the display. 

1 2. A system according to claim 1 wherein the sensor or camera, 

2 in addition to registering the image of the pointing object, can also register at least 

3 one of (i) the image of the display and (ii) the reflection or effect that the pointing 

4 device can produce on the display. 

1 3. A system as defined by claim 1 which commands the 

2 positioning of a pointing icon on the display. 

1 4. A system according to claim 1 wherein the pointing device is a 

2 part of the human body such as a hand or a finger, or an ornament or device worn on 

3 the human body such as a glove or thimble. 

1 5. A system according to claim 1 wherein the pointing device is 

2 used to point to regions of the display by way of changing its position, 

3 attitude, or presentation. 
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6. A system according to claim 1 wherein the pointing device is 
used to define a particular point or region on the display. 
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1 7. A system according to claim 1 wherein the pointing device is 

2 used to define a vector on the plane of the display that indicates a direction 

3 and magnitude relative to or with respect to an item on the display or a 

4 region of the display. 

1 8. A system according to claim 3 wherein the pointing icon on 

2 the display can be registered by the sensor or camera. 

1 9. A system according to claim 8 which also includes a method 

2 for correcting the offsets between (i) the position of the pointing device, or reflection, 

3 or effect thereof on the display as observed by the user or by the sensor or the camera, 

4 and (ii) the position of the pointer icon on the display. 

1 10. A system as defined by claim 1 which also includes at least 

2 one of the following: 

3 a. a method for selecting or highlighting a specific item or icon 

4 on the display, 

5 b. a method for activating a specific process, program, or menu 

6 item represented on the display, and 

7 c. a method for writing, scribing, drawing, highlighting, 

8 annotating, or otherwise producing marks on the display. 

1 1 1 . A method for detecting the pointing device comprising 

2 a. retrieval of data or image from a sensor or camera, and 

3 b. analysis of the data or image from the sensor or camera to 

4 locate the pointing device in the data, or locating at least a set of the picture elements 

5 in the image that comprise the rendition of the pointing device. 

1 12. A method according to claim 1 1 wherein the characteristics 

2 that distinguish the pointing device from other objects in the data from the 
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3 sensor or the image from the camera are known a priori. 

1 13. A method according to claim 1 1 wherein the characteristics 

2 that distinguish the pointing device from other objects in the data from the sensor or 

3 the image from the camera are determined based analysis of at least one set of the 

4 data acquired from the sensor or one image acquired from the camera. 

1 14. A method according to claim 1 3 wherein the characteristics 

2 that distinguish the pointing device from other objects, whose rendition are present in 

3 the data from the sensor or in the image from the camera, is obtained by 

4 a. acquiring at least two sets of data from the sensor or at least 

5 two images from the camera, one with the pointing device in view of the sensor 

6 or the camera and one without, and 

7 b. comparing the two sets with one another. 

1 1 5. A method according to claim 1 1 wherein adjustments or 

2 modifications are made to the position, sensitivity, and other settings of the sensor or 

3 the camera pursuant the analysis of the data or image retrieved from the sensor or the 

4 camera. 

1 1 6. A method according to claim 1 1 wherein at least part of the 

2 procedures for the method is carried out using at least in part the computing 

3 mechanisms available on one or more of the following: the display, or the sensor or 

4 camera, or the pointing device, or the device producing the signal shown on the 

5 display, or the device producing the pointing icon on the display. 

1 17. A method for establishing the mapping between the set of 

2 positions that a pointing device can take and the set of corresponding locations on the 

3 display comprising: 

4 a. defining the range of positions that the pointing device can 
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5 assume, 

6 b. defining the boundaries of the range of positions that the 

7 pointing device can take with geometric representations, 

8 c. transforming the geometric representation of the arrangement 

9 of regions on the display so that it fits optimally into the boundaries of the range of 
1 0 positions that the pointing device can take. 

1 1 8. A method according to claim 1 7 wherein the range of positions 

2 that the pointing device may assume is defined by querying the user to point to a set 

3 of points on the display. 

1 1 9. A method according to claim 1 8 wherein the range of positions 

2 that the pointing device can assume is defined by the boundary contours of the 

3 display as they are registered by the sensor or the camera. 

1 20. A method according to claim 19 wherein at least one special 

2 display image is used to establish the mapping between the positions that a pointing 

3 device can take and a corresponding locations on the display. 

1 2 1 . A method according to claim 1 7 wherein at least part of the 

2 procedures for the method is carried out using at least in part the computing 

3 mechanisms available on one or more of the following: the display, or the sensor or 

4 camera, or the pointing device, or the device producing the signal shown on the 

5 display, or the device producing the pointing icon on the display. 

1 22. A method for detecting the display comprising 

2 a. retrieval of data or image from a sensor or camera, and 

3 b. analysis of the data or image from the sensor or camera to 

4 locate the display in the data, or locating at least a set of the picture elements in 

5 the image that comprise the rendition of the display in the image. 
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1 23. A method according to claim 22 wherein the characteristics 

2 that distinguish the display from other objects in the data from the sensor or the 

3 image from the camera are known a priori. 

1 24. A method according to claim 22 wherein the characteristics 

2 that distinguish the display from other objects in the data from the sensor or the 

3 image from the camera are determined based on analysis of at least one set of the data 

4 acquired from the sensor or one image acquired from the camera. 

1 25. A method according to claim 22 wherein the display refers to 

2 the range of positions that the pointing device can take. 

1 26. A method according to claim 24 wherein the characteristics 

2 that distinguish the display from other objects, whose rendition are present in the data 

3 from the sensor or in the image from the camera, is obtained by 

4 a. acquiring at least two sets of data from the sensor or at least 

5 two images from the camera, one with the display off in view of the sensor or the 

6 camera and one with the display on, and 

7 b. comparing the two sets with one another. 

1 27, A method according to claim 22 wherein adjustments or 

2 modifications are made to the position, sensitivity, and other settings of the sensor or 

3 the camera pursuant the analysis of the data or image retrieved from the sensor or the 

4 camera. 

1 28. A method according to claim 22 wherein at least part of the 

2 procedures for the method is carried out using at least in part the computing 

3 mechanisms available on one or more of the following: the display, or the sensor or 

4 camera, or the pointing device, or the device producing the signal shown on the 
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display, or the device producing the pointing icon on the display. 
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